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Abstract 

The human brain is autonomously active, being characterized by a self-sustained 
neural activity which would be present even in the absence of external sensory 
stimuli. Here we study the interrelation between the self-sustained activity in au- 
tonomously active recurrent neural nets and external sensory stimuli. 

There is no a priori semantical relation between the influx of external stimuli and 
the patterns generated internally by the autonomous and ongoing brain dynamics. 
The question then arises when and how are semantic correlations between internal 
and external dynamical processes learned and built up? 

We study this problem within the paradigm of transient state dynamics for the 
neural activity in recurrent neural nets, i.e. for an autonomous neural activity char- 
acterized by an infinite time-series of transiently stable attractor states. We propose 
that external stimuli will be relevant during the sensitive periods, viz the transition 
period between one transient state and the subsequent semi-stable attractor. A dif- 
fusive learning signal is generated unsupervised whenever the stimulus influences 
the internal dynamics qualitatively. 

For testing we have presented to the model system stimuli corresponding to the 
bars and stripes problem. We found that the system performs a non-linear indepen- 
dent component analysis on its own, being continuously and autonomously active. 
This emergent cognitive capability results here from a general principle for the neu- 
ral dynamics, the competition between neural ensembles. 

Key words: recurrent neural networks, autonomous neural dynamics, transient 
state dynamics, emergent cognitive capabilities 
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1 INTRODUCTION 



It is well known that the brain has a highly developed and complex self- 
generated dynamical neural activity. We are therefore confronted with a di- 
chotomy when attempting to understand the overall functioning of the brain 
or when designing an artificial cognitive system: A highly developed cognitive 
system, such as the brain [lj , is influenced by sensory input but it is not driven 
directly by the input signals. The cognitive system needs however this sensory 
information vitally for adapting to a changing environment and survival. 

In this context we then want to discuss two mutually interrelated questions: 

• Can we formulate a meaningful paradigm for the self-sustained internal 
dynamics of an autonomous cognitive system? 

• How is the internal activity influenced by sensory signals, viz which are the 
principles for the respective learning processes? 

We believe that these topics represent important challenges for research in 
the field of recurrent neural networks and the modeling of neural processes. 
From an experimental point of view we note that an increasing flux of results 
from neurobiology supports the notion of quasi-stationary spontaneous neural 
activity in the cortex [1,2 ,3 ,4 5.6]. It is therefore reasonable to investigate the 
two questions formulated above with the help of neural architectures centrally 
based on the notion of spontaneously generated transient states, as we will do 
in the present investigation using appropriate recurrent neural networks. 

1.1 Transient- state and competitive dynamics 

Standard classification schemes of dynamical systems are based on their long- 
time behavior, which may be characterized, e.g., by periodic or chaotic tra- 
jectories [8]. The term 'transient-state dynamics' refers, on the other hand, 
to the type of activity occurring on intermediate time scales, as illustrated in 
Fig. [H A time series of semi-stable activity patterns, also denoted transient 
attractors, is characterized by two time scales. The typical duration t tr ans of 
the activity plateaus and the typical time At needed to perform the transition 
from one semi-stable state to the subsequent one. The transient attractors 
turn into stable attractors in the limit t trans /At — > oo. 

Transient state dynamics is intrinsically competitive in nature. When the cur- 
rent transient attractor turns unstable the subsequent transient state is se- 
lected by a competitive process. Transient-state dynamics is a form of 'multi- 
winners-take-all' process, with the winning coalition of dynamical variables 
suppressing all other competing activities. 
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Fig. 1. Schematic illustration of general or fluctuating activity patterns (top) and 
of transient-state dynamics (bottom), which is characterized by typical time scales 
ttrans and At for the length of activity-plateau and of the transient period respec- 
tively. 

Humans can discern about 10-12 objects per second [7j and it is therefore 
tempting to identify the cognitive time scale of about 80- 100ms with the du- 
ration t trans of the transient-state dynamics illustrated in Fig. [TJ Interestingly, 
this time scale also coincides with the typical duration [4] of the transiently 
active neural activity patterns observed in the cortex [2T3|5ll6] . 

Several high-level functionalities have been proposed for the spontaneous neu- 
ral brain dynamics. Edelman and Tononi pfTU] argue that 'critical reentrant 
events' constitute transient conscious states in the human brain. These 'states- 
of-mind' are in their view semi-stable global activity states of a continuously 
changing ensemble of neurons, the 'dynamic core'. This activity takes place 
in what Dehaene and Naccache [TT] denote the 'global workspace'. The global 
workspace serves, in the view of Baars and Franklin [12], as an exchange plat- 
form for conscious experience and working memory. Crick and Koch [13] and 
Koch [H] have suggested that the global workspace is made-up of 'essential 
nodes', i.e. ensembles of neurons responsible for the explicit representation of 
particular aspects of visual scenes or other sensory information. 

1.2 Autonomously active recurrent neural nets 

Traditional neural network architectures are not continuously active on their 
own. Feedforward setups are explicitly driven by external input [TS] and Hop- 
field-type recurrent nets settle into a given attractor after an initial period of 
transient activities |16j . The possibilities of performing cognitive computation 
with autonomously active neural networks, the route chosen by nature, are 
however investigated increasingly [T7j. In this context the time encoding of 
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Fig. 2. Illustration of clique encoding. Note that cliques are fully connected sub- 
graphs of maximal size. The dashed lines are highlighted examples of specific cliques. 
Left: A 7-site network with cliques (1,2,3), (3,6), (1,2,4,5), (4,5,6), (0,6) and (0,1). 
Right: A regular 20-site networks containing 10 four-site cliques. 

neural information, one of the possible neural codes [IB] , has been studied in 
various contexts. Two network architectures, the echo state network suitable 
for rate-encoding neurons [TH], and the liquid state machine suitable for spiking 
neurons [20J, have been proposed to transiently encode in time a given input 
for further linear analysis by a subsequent perceptron. Both architectures, the 
echo-state network and the liquid-state machine, are examples of reservoir 
architectures with fading memories, which however remain inactive in the 
absence of sensory input. 

An example of a continuously active recurrent network architecture is the 
winnerless competition based on stable heteroclinic cycles [2T]. In this case 
the trajectory moves along heteroclines from one saddle point to the next 
approaching a complex limiting cycle. Close to the saddle points the dynamics 
slows down leading to well defined transiently active neural activity patterns. 



2 CLIQUE ENCODING IN RECURRENT NETWORKS 



In order to study the issues raised in the introduction, the notion of au- 
tonomous neural activity and its relation to the sensory input, we will consider 
a specific model based on clique-encoding recurrent nets. The emphasis will be 
on the discussion of the general properties and of the underlying challenges. 
We will therefore present here an overview of the algorithmic implementation, 
referring in part to the literature for further details. 
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2.1 Cliques, attractors and transient states 



Experimental evidence indicates that sparse neural coding is an important op- 
erating principle in the brain, as it minimizes energy consumption, maximizes 
storage capacity and contributes to make information encoding spatially ex- 
plicit [22]. A powerful form of sparse coding is multi-winners-take-all encoding 
in the form of cliques. The term cliques stems from network theory and de- 
notes subgraphs which are fully interconnected [S] , a few examples are given in 
Fig. [2j Cliques are fully interconnected subgraphs of maximal size, in the sense 
that they are not part of another fully interconnected subgraph containing a 
larger number of vertices. 

Clique encoding is an instance of sparse coding with spatially overlapping 
memory states. The use of clique encoding is in fact motivated by experimental 
findings indicating a hierarchical organization of overlapping neural clique 
assemblies for the real-time memory representation in the hippocampus |23j . 
In the framework of a straightforward auto-associative neural network the 
cliques are defined by the network of the excitatory connections, which are 
shown as lines in Fig. [2], in the presence of an inhibitory background [24125] . In 
this setting all cliques correspond to attractors of the network, viz to spatially 
explicit and overlapping memory representations. 

One can transform the attractor network with clique encoding into a contin- 
uously active transient-state network, by introducing a reservoir variable for 
every neuron. In this setting the reservoir of a neuron is depleted whenever the 
neuron is active and refilled whenever the neuron is inactive. Via a suitable 
local coupling between the individual neural activity and reservoir variables 
a well defined and stable transient state dynamics is obtained [24.25J. When 
a given clique becomes a winning coalition, the reservoirs of its constituting 
sites are depleted over time. When fully depleted the winning coalition be- 
comes unstable and the subsequent winning coalition is activated through a 
competitive associative process, leading to an ever ongoing associative thought 
process. The resulting network architecture is a dense and homogeneous asso- 
ciative network (dHan) [24J. An illustrative result of a numerical simulation is 
given in Fig. [31 

For the isolated system, not coupled to any sensory input, this associative 
thought process has no semantic content, as the transient attractors, the 
cliques, have none. The semantic content can be acquired only by coupling 
to a sensory input and by the generation of correlation between the transient 
attractors and patterns extracted from the input data stream. 
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Fig. 3. Transient state dynamics of the 7-site network illustrated in Fig. [2J 
Shown are, vertically displaced, the time developments of the respective 
neural activities (solid lines) and of the neural reservoirs (dashed lines). 
The time series of spontaneously generated transient states are the cliques 
(4, 5, 6) -> (1, 2, 3) (0, 6) -» (1, 2, 4, 5) (from the left to the right). 

2.2 Competitive dynamics and sensitive periods 

To be definite, we utilize a continuous-time formulation for the dHan architec- 
ture, with rate-encoding neurons, characterized by normalized activity levels 
Xi G [0,1]. One can then define, via 



the respective growth rates r^. Representative time series of growth rates r{ 
are illustrated in Fig. [H When > 0, the respective neural activity X, in- 
creases, approaching rapidly the upper bound; when rj < 0, it decays to zero. 
The model is specified [21122]) by providing the functional dependence of the 
growth rates with respect to the set of activity-levels {xj} of all sites and on 
the synaptic weights, as usual for recurrent or auto-associative networks. 

During the transition periods many, if not all, neurons will enter the com- 
petition to become a member of the new winning coalition. The competition 
is especially pronounced whenever most of the growth rates r$ are small in 
magnitude, with no subset of growth rates dominating over all the others. 
Whether this does or does not happen depends on the specifics of the model 
setup. In Fig. HJ two cases are illustrated. In the first case (lower graph) the 
competition for the next winning coalition is restricted to a subset of neurons, 
in the second case (upper graph) the competition is network- wide. When most 
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Fig. 4. The growth rates rj(i) generating an internal transient state-dynamics via 
Eq. ([!]). The two examples differ in the functional dependence of the rj(i) on the 
Xj(t). The top graph corresponds to a system having sensitive periods, the bottom 
graph to a system without distinctive sensitive periods. 



neurons participate in the competition process for a new winning coalition the 
model will have 'sensitive periods' during the transition times and it will be 
able to react to eventual external signals. 



2.3 Sensitive periods and learning 



So far we have discussed in general terms the properties of isolated models 
exhibiting a self-sustained dynamical behavior in terms of a never-ending time 
series of semi-stable transient states, as illustrated in Figs. [3] and HI using the 
dHan architecture with continuous-time and rate-encoding neurons. 

The importance of sensitive periods comes in when the network exhibiting 
transient-state dynamics is coupled to a stream of sensory input signals. It is 
reasonable to assume, that external input signals will contribute to the growth 
rates via 

n = rf Han + An ( 1 - 6(rf a ")6(-Ar,) ) . (2) 



Here the Arj encode the influence of the input signals and we have denoted 
now with rf Han the contribution to the growth rate a neuron in the dHan layer 
receives from the other dHan neurons. The factor (1 — Q(rf Han )Q(—Ari)) in 
Eq. (j2j) ensures that the input signal does not deactivate the current winning 
coalition as we will discuss further below. Let us here assume for a moment, 
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as an illustration, that the input signals are suitably normalized, such that 
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(3) 



in order of magnitude. For the simulations presented further below a qualita- 
tively similar optimization will occur homeostatically. For the transient states 
the rf Han ~ — 1 for all sites not forming part of the winning coalition and 
the input signal Ar^ will therefore not destroy the transient state, compare 
Figs. H] and [51 With the normalization given by Eq. ([3]) the total growth rate 
~ (Ar.j — 1) will remain negative for all inactive sites and the sensory input 
will not be able to destroy the current winning coalition. The input signal 
will however enter the competition for the next winning coalition during a 
sensitive period, providing an additional boost for the respective neurons. 

This situation is exemplified in Fig. [51 where we present simulation results 
for the 7-site system shown in Fig. [21 subject to two sensory inputs Arj(t). 
The self-generated time series of winning coalitions is not redirected for the 
first sensory input. The second stimulus overlaps with a sensory period and its 
strongest components determine the new winning coalitions. The simulation 
results presented in Fig. [5] therefore demonstrate the existence of well defined 
time-windows suitable for the learning of correlations between the input signal 
and the intrinsic dynamical activity. The time windows, or sensitive periods, 
are present during and shortly after a transition from one winning coalition to 
the subsequent. A possible concrete implementation for this type of learning 
algorithm will be given further below. 

Let us now come to the factor (l—Q(rf Han )Q(—Ari)) in Eq. ([2]), containing the 
Heaviside-step functions Q(x). For vertices i of the current winning coalition 
the intra dHan-layer growth rates are positive, rf Han > 0. Therefore, the above 
factor ensures, that a suppressive Arj < has no effect on the members of the 
current winning coalition. The contribution Ar« from the input may therefore 
alter the balance in the competition for the next winning coalition during the 
sensitive periods, but not suppress the current active clique. 

Let us note, that the setup discussed here allows the system also to react to 
an occasional strong excitatory input signal having Ar^ > 1. Such a strong 
signal would suppress the current transient state altogether and impose itself. 
This possibility of rare strong input signals is evidently important for animals 
and would be, presumably, also helpful for an artificial cognitive system. 
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Fig. 5. Results for the activities Xi(t), the total growth rates ri{t) and the input 
signals Arj(i) from a simulation of the 7-site system shown in Fig. [2] (color coding). 
The time series of winning coalitions is given. The first input signal does not change 
the composition of the winning coalition, whereas the second does. 



2.4 Diffusive learning signals 



Let us return to the central problem inherent to all systems reacting to input 
signals and having at the same time a non-trivial intrinsic dynamical activity. 
Namely, when should learning occur, i.e. when should a distinct neuron become 
more sensitive to a specific input pattern and when should it suppress its 
sensibility to a sensory signal. 

The framework of competitive dynamics developed above allows for a straight- 
forward solution of this central issue: Learning should occur exclusively when 
the input signal makes a qualitative difference, viz when the input signal devi- 
ates the transient-state process. For illustration let us assume that the series 
of winning coalitions is 

(4,5,6) ^ (0,1) ^ (1,2,4,5), 

where the index [a] indicates that the transition is driven by the autonomous 
internal dynamics and that the series of winning coalitions take the form 

(4,5,6) ^ (0,1) ^ (3,6), 

in the presence of a sensory signal [s], as it is the case for the data presented 
in Fig. [5j Note, that a background of weak or noisy sensory input could be 
present in the first case, but learning should nevertheless occur only in the 
second case. A reliable distinction between these two cases can be achieved 
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via a suitable diffusive learning signal S(t). It is activated whenever any of 
the input contributions Arj changes the sign of the respective growth rates 
during the sensitive periods, 
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otherwise 



(r* > 0) and (r° 



.dHan 



<0) 



(4) 



viz when it makes a qualitative difference. Let us remember that the rf 
are the internal contributions to the growth rate, i.e. the input a dHan neuron 
receives via recurrent connections from the other dHan neurons. The diffusive 
learning signal S is therefore increasing in strength only when a neuron is 
activated externally, but not when activated internally, with the ^%ff > 
denoting the respective growth and decay rates. The diffusive learning signal 
S(t) is a global signal and a sum Ya over a ^ dynamical variables is therefore 
implicit on the right-hand side of Eq. (|4]). 



2. 5 The role of attention 

The general procedure for the learning of correlation between external signals 
and intrinsic dynamical states for a cognitive system presented here does not 
rule out other mechanisms. Here we concentrate on the learning algorithm 
which occurs automatically, one could say sub-consciously. Active attention 
focusing, which is well known in the brain to potentially shut off a sensory 
input pathway, or to enhance sensibility to it, may very well work in parallel 
to the continuously ongoing mechanism investigated here. 

We note, however, that the associative thought process within the dHan carries 
with it a dynamical attention field [23]. Neurons receiving both positive and 
negative contributions from the winning coalition will need smaller sensory 
input signals in order to be activated than neurons receiving only negative 
contributions. To put it colloquially: When thinking of the color blue it is 
easier to spot a blue car in the traffic than a white one. 



1 The name 'diffusive learning signal' [8] stems from the fact, that many neuro- 
modulators are released in the brain in the intercellular medium and then diffuse 
physically to the surrounding neurons, influencing the behavior of large neural as- 
semblies. 
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Fig. 6. Schematic representation of the information flow from a raw pattern (bot- 
tom) via the input layer (middle) to the dHan layer (top). The synaptic strengths 
Vij connecting the input with the dHan layer are adapted during the learning pro- 
cess (illustrated selectively by respective thick/thin blue lines in the graph). The 
dHan layer consists of active and inactive neurons (red/yellow circles) connected by 
intra-layer synaptic weights. The topology shows five cliques (denoted C-I to C-V 
in the graph) of which C-II is active, as emphasized by the red-color neurons. 

3 COMPETIVE LEARNING 

So far we have described, in general terms, the system we are investigating. It 
has sensitive periods during the transition periods of the continuously ongoing 
transient-state process, with learning of input signals regulated by a diffusive 
learning signal. The main two components are therefore the dHan layer and 
the input layer, as illustrated in Fig. El 

3.1 Input data-stream analysis 

The input signal acts via Eq. (j2J) on the dHan layer, with the contribution Ar^ 
to the growth rate of the dHan neuron i given by 

Ari = J2 VijVj, As i = %(! - Vj), (5) 

3 3 

where we have denoted with yj e [0, 1] the activity-levels of the neurons in 
the input layer. For subsequent use we have defined in Eq. (J5]) an auxiliary 
variable As*, which quantifies the influence of inactive input-neurons. The task 
is now to find a suitable learning algorithm which extracts relevant information 
from the input-data stream by mapping distinct input-patterns onto selected 



11 



winning coalitions of the dHan layer. This setup is typical for an independent 
component analysis [26] . 



The multi-winners-take-all dynamics in the dHan module implies individual 
neural activities to be close to 0/1 during the transient states and we can 
therefore define three types of inter-layer links Vij (see Fig. [6]): 

• active ( 'act) 

Links connecting active input neurons with the winning coalition of the 
dHan module. 

• orthogonal ( 'orth ') 

Links connecting inactive input neurons with the winning coalition of the 
dHan module. 

• inactive ( 'ina ') 

Links connecting active input neurons with inactive neurons of the dHan 
module. 

The orthogonal links take their name from the circumstance that the receptive 
fields of the winning coalition of the target layer need to orthogonalize to all 
input-patters differing from the present one. Note that it is not the receptive 
field of individual dHan neurons which is relevant, but rather the cumulative 
receptive field of a given winning coalition. 

We can then formulate three simple rules for the respective link-plasticity. 
Whenever the new winning coalition in the dHan layer is activated by the input 
layer, viz whenever there is a substantial diffusive learning signal, i.e. when 
Sdiff exceeds a certain threshold S^ft, the following optimization procedures 
should take place: 

• active links 

The sum over active afferent links should take a large but finite value r" ci , 



_, r act 
v 

Xi active 



£ V V Vj 
3 

orthogonal links 

The sum over orthogonal afferent links should take a small value s° rth , 



orth 



Xi active 



inactive links 

The sum over inactive links should take a small but non-vanishing value 

r ina 
v ) 

J2 v a Vj 



Xi inactive 
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The r" ci , r* na and s° rth are the target values for the respective optimization 
processes. In order to implement these three rules we define three correspond- 
ing contributions to the link plasticities: 

c act = pact q( x . _ x act^ g ign ( r act - Ar t ) 

c orth = T orth Qfa _ x actj Sig n ( s ^th _ Ag .) (6) 
c ma = T ina Q^ina _ ^ Sig n ( r «™ _ & r .) 



where the inputs Ar^ and Asi to the dHan layer are defined by Eq. (j3J). For the 
sign-function Sign(x) = ±1 is valid, for x > and x < respectively, Q(x) 
denotes the Heaviside-step function. In Eq. (jB]) the T® ct , Y° rth and T* na are 
suitable optimization rates and the x^ ct and x™ a the activity levels defining 
active and inactive dHan neurons respectively. A suitable set of parameters, 
which has been used for the numerical simulations, is given in Table HJ 

Using these definitions, the link plasticity may be written as 

jv %3 = e(s diff - s c diff ) [ cf y 3 + c? th (i - Vj ) + er Vj } , (7) 



where S^i// is an appropriate threshold for the diffusive learning signal. The 
inter-layer links Vij cease to be modified whenever the total input is optimal, 
viz when no more 'mistakes' are made [27]. 

We note, that a given interlayer-link is in general subject to competitive 
optimization from the three processes (act/orth/ina). Averaging would occur 
if the respective learning rates T^ ct /T° rth /T^ a would be of the same order of 
magnitude. It is therefore necessary, that T^ ct ^> Y° rth and T" ct 3> r* na . 



3.2 Homeostatic normalization 



It is desirable that the interlayer connections neither grow unbounded with 
time (runaway-effect) nor disappear into irrelevance. Suitable normalization 
procedures are therefore normally included explicitly into the respective neural 
learning rules and are present implicitly in Eqs. ([6]) and Eq. (jTJ. 

Table 1 

The set of parameters entering the time-evolution equations of the links connecting 
the input to the dHan layer, with r^j// = ^-0 an d ^diff = 0-15, used in the actual 
simulations. 
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Fig. 7. Examples of typical input patterns for a 5 x 5 bars problem with a probability 
p = 0.1 for the occurrence of the individual horizontal or vertical bars. The prob- 
lem is non-linear since the pattern intensity is not enhanced when an elementary 
horizontal and vertical bar overlap each other. 



The strength of the input-signal is optimized by Eq. (j7J) both for active as well 
as for inactive dHan neurons, a property referred to as fan-in normalization. 
Eqs. OH]) and (|7j) also regulate the overall strength of inter-layer links emanating 
from a given input layer neuron, a property called fan-out normalization. 

Next we note, that the time scales for the intrinsic autonomous dynamics in the 
dHan layer and for the input signal could in principle differ substantially. Po- 
tential interference problems can be avoided when learning is switched-on very 
fast. In this case the activation and decay rates ^iff f° r ^ ne diffusive learning 
signal are large and the corresponding characteristic time scales \/T di ff are 
smaller than both the typical time scales of the input and of the self-sustained 
dHan dynamics. 



4 THE BARS PROBLEM 



A cognitive system needs to extract autonomously meaningful information 
about its environment from its sensory input data stream via signal separation 
and feature extraction. The identification of recurrently appearing patterns, 
i.e. of objects, in the background of fluctuation and of combinations of distinct 
and noisy patterns, constitutes a core demand in this context. This is the do- 
main of the independent component analysis [26] and blind source separation 
|28j . which seeks to find distinct representations of statistically independent 
input patterns. 

In order to test our system made-up by an input layer coupled to a dHan 
layer, as illustrated in Fig. E] we have selected the bars problem [2"§f50] . The 
bars problem constitutes a standard non-linear reference task for the feature 
extraction via a non-linear independent component analysis for an L x L input 
layer. Basic patterns are the L vertical and L horizontal bars. The individual 
input patterns are made-up of a non-linear superposition of the 2L basic bars, 
containing with probability p = 0.1 any one of them, as illustrated in Fig. [7J 
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Fig. 8. For the 5x5 bars problem the response, as defined by Eq. ([8]), for the 
10 winning coalitions in the dHan layer (compare Fig. [2] and Fig. [6]) to the ten 
reference patterns, viz the 5 horizontal bars and the 5 vertical bars of the 5x5 
input field. In the top row the numbering of the cliques C-I,..,C-X having the max- 
imal response to the respective reference patterns is given. In the bottom row, 
below each of the 10 black/white reference patterns, the receptive fields, Eq. ([9]), 
for the winning coalitions C-I,..,C-X given in the top row are given color-coded, 
with black/blue/red/yellow coding synaptic strengths of increasing intensities. 

4-1 Simulations and setup 

For the simulations we presented to the system about N pat t ~ 5 x 10 3 randomly 
generated 5x5 input patterns of the type shown in Fig. [7J The bars pattern are 
black/white with the yi = 1/0 for active/inactive sites, irrespectively of pos- 
sible overlaps of vertical and horizontal bars. The individual patterns lasted 
Tpatt = 20 with about T inter = 100 for the time between two successive in- 
put signals. These time scales are to be compared with the time scale of the 
autonomous dHan dynamics illustrated in the Figs. H] and [5j for which the 
typical stability-period for a transient state is about t tra ns ~ 70. We also note 
that there is no active training for the system. The associative thought pro- 
cess continuous in the dHan layer, at no time are the neural activities reset 
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and the system restarted. All that happens is that the ongoing associative 
thought process is influenced from time to time by the input layer and that 
then the synaptic strengths Vij connecting the input layer to the dHan layer 
are modified via Eq. ([7]). 

The results for the simulation are presented in Fig. [SJ For the geometry of the 
dHan network we used a regular 20-site star containing 10 cliques, with every 
clique being composed of four neurons, see Fig. [2j In Fig. [8] we present the 
response 



1 v-^ a a = 1, .., 10 

5 ^aJ ieCa ,j /3 = 1, .., 10 



of the 10 cliques C a in the dHan layer to the 10 basic input patterns {yj,j = 
1, ..,25}, the isolated bars. Here the C a G {C-I,..,C-X} denotes the set of 
sites of the winning-coalition a and S{C a ) its size, here S(C a ) = 4. The 
response R(a,j3) is equivalent to the clique averaged afferent synaptic signals 
Arj, compare Eq. (jSJ), in the presence of an elementary bar in the sensory 
input field. 



4-2 Semantic learning 



The individual potential winning coalitions, viz the cliques, have acquired in 
the course of the simulation, via the learning rule Eq. (|7D, distinct suscep- 
tibilities to the 10 bars, compare Fig. At the start of the simulation the 
winning coalitions were just given by properties of the network typology, viz 
by the cliques, having no explicit semantic significance. The susceptibilities to 
the individual bars, which the cliques have acquired via the competition of the 
internal dHan dynamics with the sensory data input stream, can then be inter- 
preted as a semantic assignment. The internal associative thought process of 
the dHan layer therefore becomes semantically meaningful via the coupling to 
the environment, corresponding to a sequence of horizontal and vertical bars. 
This learning paradigm is compatible with multi-electrode array studies of the 
visual cortex of developing ferrets p], which indicate that the ongoing corti- 
cal dynamics is void of semantic content immediately after birth, acquiring 
semantic content however during the adolescence. 



16 




C-l 




C-ll 




C-lll 




C-IV 


-□ 


C-V 


■ 


C-VI 




C-VII 


- * - 


C-VI II 


- A- 


C-IX 




C-X 




C-XI 


T 


C-X II 


--0-- 


C-XI II 


♦ 


C-XIV 




c-xv 




C-XVI 


-©-- 


C-XVII 


-e- 


C-XVI II 


-e- 


C-XIX 




c-xx 


-©- 



Fig. 9. For the 10 x 10 bars problem the response, as defined by Eq. (jSj), for the 20 
winning coalitions C-I,..,C-XX in the dHan layer (compare Fig. [2] and Fig. [6]) to the 
twenty reference patterns, viz the 10 horizontal bars and the 10 vertical bars of the 
10 x 10 input field. 



4-3 Competitive learning 



The winning coalitions of the dHan layer are overlapping and every link Vij 
targets in general more than one potential winning coalition in the dHan 
layer. This feature contrasts with the 'single-winner-takes-all' setup, normally 
used for standard neural algorithms performing an independent component 
analysis |26j, for which the target neurons are physically separated. For the 
regular 20-site network used in the simulation every dHan neuron appertains 
to exactly two cliques, compare Fig. [2j The unsupervised learning procedure, 
Eq. involves therefore a competition between the contribution c" ct , c orth 
and c ma , as given by Eq. ([6]). For the simulations we used a set of parameters, 
see Table [TJ for which the contribution to c a { ct is adapted at a much higher 
rate than the contributions to c orth and c ma . The responses R(a,/3) of the 
winning coalitions are therefore close to, but somewhat below, the optimal 
value r° ci = 0.8 used for the simulations, compare Fig. [HI The target value r% ct 
will not be reached even for extended simulations, due to the competition with 
the other optimization procedures, namely c orth and c ma , compare Eq. 
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Fig. 10. For the 10 x 10 bars problem the color-coded receptive fields, 
Eq. ©, for the 20 cliques C-I,..,C-XX of the dHan layer, compare Fig. [9j with 
black/blue/red/yellow coding synaptic strengths of increasing intensities. 



4-4 Receptive fields 



The averaged receptive fields 



«ec Q 



a = 1, .., 10, 



(9) 



of the a = 1, 10 cliques in the dHan layer with respect to the j = 1, 25 
input neurons are also presented in Fig. [HJ The inter-layer synaptic weights 
Vij can be both positive and negative and the orthogonalization procedure, 
Eq. ([6]), results in complex receptive fields. The time evolution equations for 
the inter-layer synaptic strengths (jTJ are optimizing, but not maximizing, the 
response of the winning coalition to a given input signal. The receptive fields 
retain consequently a certain scatter, since the optimization via Eq. (JTJ ceases 
whenever a satisfactory signal separation has been obtained. This behavior is 
consistent with the 'learning by mistakes' paradigm [27], which states that a 
cognitive system needs to learn in general only when committing a mistake. 
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4-5 Emergent cognitive capabilities 



The simulation results for the 5x5 bars problem presented in Fig. [8] may be 
generalized to larger systems. For comparison we discuss now the results for 
the 10 x 10 bars problem, for which there are ten horizontal and ten vertical 
elementary bars. For the dHan network we used a regular 40-site star with 
20 cliques, a straightforward generalization of the regular star illustrated in 
Fig. [2j We used otherwise exactly the same set of parameters as previously for 
the 5x5 bars problem, in particular also the same number of input training 
patterns. No optimization of parameters has been performed. The respective 
responses R(a,j3) and receptive fields F(a,j) (compare Eqs. ([HD and are 
presented in Fig. and [TU1 



The probability for any of the 20 bars to occur in a given input pattern, like 
the ones for the 5x5 bars problem illustrated in Fig. [7J is p = 0.1 and any 
individual 10 x 10 input patterns contains on the average m 2.3 bars superposed 
non-linearly. The separation of the 20 statistically independent components in 
the input data stream is therefore a non-trivial task. The results presented 
in Fig. M indicate that the system performs the source separation surprisingly 
well, but not perfectly. The respective receptive fields, shown in Fig. [TDJ are 
only in part self-evident. This is, again, due to the competitive nature of the 
unsupervised and local learning process, which has the task to optimize the 
input rates to the dHan layer and not to maximize the signal-to-noise ratio. 
We note in this context that the system contains no prior knowledge about 
the nature and statistics of the input signals. 



In fact, the system has not been constructed in the first place to tackle the non- 
linear independent component task. The setup used here has been motivated 
by two simple guiding principles, the occurrence of self-sustained internal neu- 
ral activity and the principle of competitive neural dynamics. These principles 
have been used in our study to examine the interplay of the self-sustained in- 
ternal neural dynamics with the inflow of external information via a sensory 
data stream. One can therefore interpret, to a certain extend, the capability 
of the system to perform a non-linear independent component analysis as an 
example of an 'emergent cognitive capability'. This information processing ca- 
pability emerges from general construction principles and does not result from 
the implementation of a specific neural algorithm. 



19 



5 DISCUSSION AND CHALLENGES 



5. 1 Discussion 



A standard approach in the field of neural networks is to optimize the design 
of a network such that a given cognitive or computational task can be tack- 
led efficiently. This strategy has been very successful in the past with respect 
to technical applications like handwriting recognition [31] and regarding the 
modeling of initial feed-forward sensory information processing in cortical ar- 
eas like the primary optical cortex [32]. Task-driven network design standardly 
results in input-driven neural networks, with cognitive computation coming 
to a standstill in the absence of sensory inputs. 

Real-world cognitive systems like the human brain are however driven by 
their own internal dynamics and it constitutes a challenge to present and to 
future research in the field of neural networks to combine models of this self- 
sustained brain activity with the processing of sensory data. This challenge 
regards especially recurrent neural networks, since recurrency is an essential 
ingredient for the occurrence of spontaneous internal neural activities. 

In this work we studied the interplay of self-generated neural states, the time- 
series of winning coalitions, with the sensory input for the purpose of unsuper- 
vised feature extraction. We proposed learning to be autonomously activated 
during the transition from one winning coalition to the subsequent one. 

This general principle may be implemented algorithmically in various fashions. 
Here we used a generalized neural net (dHan - dense homogeneous associative 
net) for the autonomous generation of a time series of associatively connected 
winning coalitions and controlled the unsupervised extraction of input-features 
by an autonomously generated diffusive learning signal. 

We tested the algorithm for the bars problem and found good and fast learn- 
ing and that the initially semantically void transient states acquired, through 
interaction with the data input stream, a semantic significance. Further pre- 
liminary results indicate that the learning algorithm retains functionality un- 
der a wide range of conditions and for various sets of parameters. We plan 
to extend the simulations to various forms of temporal inputs, especially to 
quasi-continuous input and to natural scene analysis, and to study the em- 
bedding of the here proposed concept within the framework of a full-fledged 
and autonomously active cognitive system. 
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5.2 The overall perspective 



There is a growing research effort trying to develop universal operating prin- 
ciples for biologically inspired cognitive systems, the rational being, that the 
number of genes in the human genome is by far too small for the detailed 
encoding of the fast array of neural algorithms the brain is capable off. There 
is therefore a growing consensus, that universal operating principles may be 
potentially of key importance also for synthetic cognitive and complex systems 
[33E1]- The present work is motivated by this line of approach. 

Universal operating principles for a cognitive system remain functionally op- 
erative for a wide range of environmental conditions. Examples are, universal 
time prediction tasks for the unsupervised extraction of abstract concepts and 
intrinsic generalized grammars from the sensory data input stream [3"oH3l)f8"] 
and the optimization of complexity and information theoretical measures for 
closed-loop sensorimotor behavioral studies of simulated robots [37.38.39j. The 
present study is motivated by a similar line of thinking, investigating the con- 
sequences of a self-sustained internal neural activity in recurrent networks, 
being based on the notion of transient-state and competitive neural dynam- 
ics. The long-term goal of an autonomous cognitive system is pursued in this 
approach via a modular approach, with each module being based on one of 
the above mentioned general architectural and operational principles. 
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